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Abstract 

We formulate two types of extension of the large deviation theory initiated by Bahadur in a non-regular 
setting. One can be regarded as a bound of the point estimation, the other can be regarded as the limit of a 
■ bound of the interval estimation. Both coincide in the regular case, but do not necessarily coincide in a non- 

' regular case. Using the limits of relative Renyi entropies, we derive their upper bounds and give a necessary 

^ and sufficient condition for the coincidence of the two upper bounds. We also show the attainability of these 

two bounds in several non-regular location shift families. 
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I/-) . 1 Introduction 



As discussed by Bahadur g, |j , Fisher information characterizes the limit of the decreasing rate of the tail 
probability of the optimal estimator. However, when the KuUback-Leibler (KL) divergence diverges or Fisher 
information cannot be defined, this cannot be applied. As an alternative information quantity between two 
■ probabilities, we can use the relative Renyi entropies := — log/ p''{uj)q-^''^{uj) duj (0 < s < 1), which 

"j^ , play an important role in simple hypothesis testing. (Hoeffding Q, Chernoff In general, for a probability 
distribution family {pe\0 C M}, the relative Renyi entropies I^^peWpe+e) tend to when e goes to 0, but their 
order is not necessarily e^. In this paper, we treat a large deviation theory, which can be applied to such a case. 
The importance of the relative Renyi entropies was pointed out by Akahira |^ from the viewpoint of 

information loss in a non-regular family. Although the limit distribution of the maximum likelihood estimator 
(MLE) in a non-regular family has been extensively discussed ||^, ^, ^, large deviation theory has not been 
, discussed sufficiently in this respect. 
' In a large deviation theory of the parameter estimation, we usually focus on the first exponential rate 
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f3{f,e,e):^lunM — \ogpn\Tn-0\>e} (1) 



. for a sequence of estimators T — {T„}, which is simply called an estimator in the following. Of course, when 
' (3{T, 9, e) is large, the estimator T is better. We discuss its maximization at the limit e 0. As is explained in 
' Section Bahadur focused on the bound a{6) of the first exponential rate at the limit e — + 0, whose definition 
is precisely given in Section ||. He derived its upper bound from the viewpoint of Stein's lemma in simple 
hypothesis testing. 

The main purpose of this paper is to extend the bound a{0) and derive its upper bound in a more general 
■ setting. Indeed, depending on how the limit e — *■ is obtained, we can extend the bound a{9) in two ways. 
^ ] This difference can be regarded as the difference between the point estimation and the limit of the interval 
- - I estimation. Bahadur's original theory concerned the point estimation, but it can also be applied to the limit 
of the interval estimation, as is explained in Section |^. Therefore, from the two ways of extending the bound 
a{9), we can define two generalizations ai{9) and a2{9) of Bahadur's bound a{9). In Section ^, we give the 
respective upper bounds ai{9) and 02(6*), and check that our result contains Bahadur's results as a special case 
in which the two upper bounds ai{9) and a2{9) coincide. Proofs of the two inequalities ai{9) < ai(9) and 
a2{9) < a2{9) are given in Section ^ 

In Sections ^ and ^, we also discuss the attainability of upper bounds ai (9) and a2 (^) , and calculate them 
for location shift families. In Section ^, we derive several formulae for the first exponential rates of the maximum 
likelihood estimator (MLE) and some estimators consisting of order statistics under certain assumptions. In 
Section ^, using a formula for relative Renyi entropies given by Hayashi pO| ] , we calculate the two upper bounds 
ai{9) and 52(^)7 and derive a sufficient and necessary condition for their coincidence. Using formulae given in 
Section ^ we check that these bounds are attainable in some special cases. Our examples in Section |^ contain 
location shift families generated by a WeibuU distribution, gamma distribution, beta distribution, and uniform 
distribution. 
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2 Bahadur theory 



In this section, we begin by summarizing the results reported by Bahadur |[ ^, who discussed the decreasing 
rate of the tail probability in the estimation for a distribution family. Given n-i.i.d. data wi, . . . , w„, the first 
exponential rate P{T, 9, e) of the estimator T — {T„} is written as 

f3{f, 9, e) = min{/3+(f , 9, e),p-{f, 9, e)}, 

where the exponential rates of half-side error probabilities are given by 

/3+(f, 9, e) := liminf — logp^{T„ > 9 + e} 
n 

13- {f, 9, e) := liminf — logp^{T„ <9^t}. 
n 

When an estimator T = {r„} satisfies the weak consistency 

Pe{\Tn-0\ >e}^0 Ve > 0, V0 G 9, 
using the monotonicity of KL-divergence, we can prove the inequality 

P{f,9,e) < mm{D{pg+,\\pg),D{pg_,\\pg)}. (2) 

Note that if, and only if, the family is exponential, there exists an estimator attaining the equality (|^) at 
y9 £ 8,Ve > 0. Therefore, for a general family, it is difficult to optimize the first exponential rate (3{T,9,e). 
We usually introduce the second exponential rate as another optimized value: 

a(f,9) := lim ^(3(f,9,e). (3) 

In this case, when the Fisher information Jg satisfies the condition 

Jg := / lg{LoYpg{(Lj) = lim —D{pg+^\\pg), lg{uj) := — logp0(u;), 
J c^o d9 

the inequality 

aif,9)<^Je (4) 

holds. Moreover, as was proven by Fu if the family satisfies the concavity of the logarithmic derivative 
le{uj) for 9 and some other conditions, the MLE 9ml attains the equality of (^. These facts are summarized in 
the two inequalities 

ai9) := sup a(f,0) = ^Jg (5) 

f :WC 

lim 1 sup /?(f,^,e) = ij9. (6) 

Inequality (|^) can be regarded as the bound of the point estimation, while inequality (|^) can be regarded as 
the limit of the bound of the interval estimation because sup^^.j^p l3{T,9,e) corresponds to the bound of the 
interval estimation with width 2e of the confidence interval. 



3 Main results 

In this paper, the relative Renyi entropies P{pg\\pg+c) substitute for the KL divergence. Note that the order of 
I'^ {pe\\pe+e) is not necessarily at the limit e ^ 0. However, its order is independent of the parameter s, as is 
guaranteed by the inequalities 

2min{s,l - s}I^ {pg\\pg+^) < I%pg\\pg+e) < 2max{s,l - s}I^pg\\pg+^), 
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which are proven in Lemma of the Appendix. In several cases, the order of the first exponential rate 
(3{T, 9, e) coincides with the order of the relative Renyi entropies /''(pe ||pe+e)- In the following, we use a strictly 
monotonically decreasing function g{x) such that /"(peHpe+e) — 0{g{e)) and 5(0) — 0. 
Following equations (||) and (||), we define two extensions of Bahadur's bound a{9) as 

ai (0) hm sup sup inf /3(f , 6', e) (7) 

e^+O g{e) T e^e<e'<6+e 

a2(^) :=supa2(f,0), (8) 

T 

where 

a2(T,6') := liminf ^ inf p(f,9',e). (9) 

Note that we take infimum inie-^<:e'<g+g into account in (^), unlike (^). As was pointed out by Ibragimov 
and Has'minskii p^ , when KL-divergence is infinite, there exists a consistent super efficient estimator T such 
that (]{T,9,e) and lim^^^Q j^P{T,9,e) is infinite at one point 9. Therefore, we need to take the infimum 
infe_e<6)'<e+e into account. In this situation, we do not need to limit estimators to weakly consistent ones. As 
is proven in the next section, we can obtain the following theorems. 

Theorem 1 When the convergence lime_>o ^ '^^0-'^'^^^''+'^^'' ig uniform for < s < 1, the inequality 

aiiO) <ai{9) := 2" sup /„% (10) 

0<s<l ' 

holds, where k and 1^ g are defined by 

r ,^ lin, I'iPo-^M\Pe^^/^) i>,>o (11) 

= lini 

e^+O g{e) 

Lemma ^ proven in Appendix guarantees the existence of such a real number k. 

Note that the function s — > /| is concave and continuous because the function s —> P{pg_i^\\pg^i^) is concave 
and continuous. Therefore, when Ig g = e^i '^6 have 

«iW = 2%V (12) 



Theorem 2 // the convergence 

hm^-^o ^'^^"'g'e)^"'"'^'^ ^« uniform for s £ (0, 1) and 9 e K for any compact set 

K cM., the inequality 



suPo<s<i^(fej(s^ + (l-s)^)" ' tfK<l 
a2{9) <a2i9) := { 2/|, tf k = 1 (13) 

info<.<i^(fej(s^ + (l-s)^)" ifK>l 



holds. 



In our proofs of these theorems, Chernoff's formula and Hoeffding's formula in simple hypothesis testing play 
important roles. 

As was proven by Akahira Q , under some regularity conditions for a distribution family, the equation 

^.^ I^Mpo^A ^ Jos{l -s)^ ^^^^ 
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holds. When we choose g{x) = x^, we have k = 2,1^ g = ^Jes(l — s), and the relations 

ai{9) = 4 max /g g = Je 

P 1 
^2(6') = min — - = -Je 
o<s<i s(l - s) 2 

hold. In particular, if the distribution family satisfies the concavity of the logarithmic derivative le{uj) for 9 and 
some other conditions, the bound ^Jg is attained by the MLE. Thus, the relations ai{6) = 02(6*) = ai{9) = 
a2{0) = \Je hold. 

As a relation between two bounds and 0.2(0) ^ we can prove the following theorem. 

Theorem 3 The inequality 

aM>a2{e) (15) 
holds, and ( |7^ holds as an equality if and only if the equations 

am = 2-Ilg (16) 
2«4 = a2(0) (17) 
hold. When < 1, ( f/^ holds as an equality if and only if equation ( fJ^ j holds. 



d TS 



0. 



When Ig g is differentiable, condition is equivalent to 

4 Proofs of main results 

In our proofs of Theorems Q and ^, Chernoff 's formula and Hoeffding's formula in simple hypothesis testing 
are essential, and are summarized as follows. Let the probability p on be the null hypothesis and q be 
the alternative hypothesis. When we discuss a hypothesis testing problem concerning n-i.i.d. data, we call a 
sequence A — {An} a test, where An is an acceptance region, which is a subset of il". The first error probability 
ei{An) and the second error probability 62 (A„) are defined as 

e,{An) := 1 -p"(A„), e2{An) := g"(A„), 

and their exponents are given by 

el (A) := lim inf — log ei (A„) 

n — >oo Ji 

62 (A) := liminf — loge2(A„). 

n — 'oo Ji 

Chernoff |^ evaluated the exponent of the sum of the two errors as 

sup lim — log(ei(A„)+e2(A„)) = supmin{e*(i*),e;(i')}= sup P{p\\q), (18) 

A " A 0<s<l 

which is essential for our proof of Theorem This bound is achieved by both of the likelihood tests {cj„ G 
> q'^iiSn)} and {iOn e r!"b"('^n) > q'^iiSn)}. 
Hoeffding proved another formula for simple hypothesis testing Q : 

sup eUA)^ sup + (19) 

A:el(A)>r <s <1 1 S 

This formula is essential for our proof of Theorem 0. 
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Proof of Theorem^ Applying equation jl^ ) to the two hypotheses pB-a and PB+e^ we obtain 



inf /3(r,(?',e) <min{/3+(r,0-e,e),/3-(r,0 + e,e)} < sup P {pe-,\\pe+.). 
e-e<e'<e+e o<s<i 



Taking the hmit e — > 0, we have 



1 



hm sup inf (3(T,e\e) < 2" sup 

e^+o g{e) f,^^e-e<e'<0+e o<s<i ^' 



In the regular case, this method was used by Sievers |1G 
Proof of Theorem Hoeffding's formula (|l^) yields 



< sup ■ 

<s <1 



< sup ■ 

0<s <1 



-./3+(T,g~^£,r;e) + /-(pe_iJbg+iJ 
1 - s 

-s inf0_,,e<e.<0+,,e /3(f, 9', rje) + /"(pg^ i i J 



The uniformity of IhWj guarantees that 



a2{T,9){l-r]r < sup 

0<s<l 



1 - s 



From (|l|), we have 

a2(T, 9) < sup ^ a 



?/ < max 



0<s<l 1 — s 



,a;,y>OL 0<V77<1 



We define the set Ci and ao as 



y > sup — - — - — 

<t <1 i — I 



< X < ao 
X > ao 



ao := sup{a|(a77'',a(l - r;)'") ^ int(Ci), < V77 < 1}. 
Note that the function s ^ I'^ g is concave. We define the convex function g{x) and another set C2 as 



C2 := {{x,y)\y > g{x)}. 

Since inequality ( p2[ ) guarantees 

(a2(T, 0)^y^ a2(r, ^)(1 - r?)") ^ int(Ci), < Vt? < 1, 

the inequality 

a2(T, 6*) < ao 

holds. Relations (p^), (|25|), and (|2^) guarantee the relation 

Ci C C2. 



Applying Lemma |9[ we have 

( -tx + iig 

inf {sx + (1 — s)y) = inf sx + {1 — s) sup 



{x,y)eCi x>0 \ 0<t<l 1—t I 

( (S - t)X + (1 - S)P n \ 

= inf sup ^ , \ - lie- (29) 

In the foUowing, we divide our situation into three cases k > 1,k = 1,1 > k > 0. When k > 1, relation 



( |28[ ) guarantees that 

ans{l - s) ( + {1 - s)^) = min (577'= + (1 - s)(l - t?)'') 
\ / <i) <i 

= inf {sx + (1 - s)?/) < inf (sx + (1 - s)y) = /%. 

(a:,y)eC2 (a:,y)eCi 



Therefore, 

K-l 

(s^ + (l-s)^' 

a2(r,0) <ao < 

which implies (|T3|). 

When K = 1, similarly, we can easily prove 



(s^ + (l-,s)^) 



ia.(T» . a.(T»^mm^ 1(1 - ,)) < ^^inf^^ (i. + i,) = /|,. 

Finally, we consider the case where k < 1. Since function g is concave on (0, ao), there exists 770 G (0, 1) 
such that 

«o(r/^,(i-??o)") eCincf. (30) 

Since Ci and C2 are convex, there exists a real number sq G (0, 1) such that 

sup (sqx + (1 - so)y) = ao(so?7o + (1 - so)(l - ?7o)'*') = hif {sqx + {1 ~ SQ)y) . 

{x.,y)£C^ (x,y)eCi 



In general, for any s e (0, 1), using (|29|), we obtain 



aos(l - s) I s»-i + (1 - s)»-i = ao sup (s??'" + (1 - s)(l - t?)*^) = sup (sx + (1 - s)?/) 

^ ^ 0<))<1 (2;,a)eCf 



>ao (s77^ + (1 - s)(l - %)") > ^ inf (sx + (1 - s)y) = I^^^, 



inf 



which lead to ( |l; 

Proof of Theorem It is trivial in the case of k = 1 . When k > 1 , it is also trivial because 

^" " (s^ + (1 - s)^y < 2%% < 2'^ sup 

^ / 0<s<l 



0<S<1 s(l — s) 
Next, we consider the case k < 1. The inequality 



sup (s^ + (1 - s)^)'"' < 2-^ sup II, (31) 

0<s<l S(l - Sj V / 0<s<l 

follows from the two inequalities 

Ile< sup I^e (32) 

0<s<l 

1 



s(l-s) 



s— + (l-s)— < 2'=. (33) 
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We thus obtain (|T^) . In the following, we prove that the equality of (^5|) implie s and ( |l^ ) implies (^7|) and 
the equality of ( |15| ) in the case where n < 1. If we assume that the equality of (|3l|) holds, the equalities of ( p2|) 
and ( |33| ) hold at the same s. The equality of ( |33| ) holds if and only if s = i. Therefore, 

^g,e = sup rig, 

0<s<l 

which is equivalent to (^6|). If we assume that ( |l6| ) holds, inequality (|3^) guarantees that 

Me) < ^^Ile- (34) 
Substituting ^ into s at the left hand side (LHS) in the definition of a2(^), we obtain 

Q2W > 2^ lie- 

Thus, equation (|l^) holds. Combining ( pj| ) and (p^), we obtain the equality of (p^). ■ 

5 First exponential rates of useful estimators 

In the following, we discuss the first exponential rate /3{T, 6, e) of a useful estimator T for a location shift family 
{f(x — d)\0 £ K}, where / is a probability density function on R. 

When the function di{x) :— f{x + e)/ f{x — e) is monotonically decreasing w.r.t. x when both f{x + e) and 

f{x ~ e) are not zero, we can define the likelihood ratio estimator 0^ := {Qn,e.[x\-, ■ ■ ■ ,a;„)}, which depends on 
the constant e > 0, as shown by 

^„,e ^ (sup{z|fc(z) < 0} + inf{z|fc(z) > 0}) , (35) 

where the monotonically decreasing function fc(z) is defined by 

1 " 

Kz) -=-^2 (log /(^^ -z + e)-logf{x,-z-e)). (36) 
1=1 

Note that when log f{x) is concave, the above condition is satisfied. This definition is well defined although the 
monotonically decreasing function k{z) is not continuous. 

If the support of / is (a, 6), we need to modify the definition as follows. In this case, we modify the 
estimator 6'„ g by using the two estimators 9n ■= max{a;i, . . . , x„} — b and := min{a;i, . . . , Xn} ~ a- When 
§.n ~ > 2e, the estimated value is defined by (|35| ) in the interval {9^ — e,0„ + e). When 0^ — On < 2e, we 
define 6n,e ■— \ (^„ + &n) ■ Moreover, when the support of / is a half fine (0, 00), the estimated value is defined 
by (HI) in the half fine (^„ — e, cxo). 

We have the following lemma. (A regular version of this lemma is discussed by Huber , Sievers jl^ , and 
Fu ll|.) 

Lemma 1 When logf{x) is concave, the equation 

mm{l3-Cl,0,e),l3+Cl,0,e)}= sup ^ (/e-,||/e+e). (37) 

<s <1 

holds, where feix) :— f{x — 0). 

Therefore, the equality of inequality ( pO| ) holds in this case. 

Proof: Note that l3+{%e,e) = l3+{%e - e,e) and l3-{%d,e) = l3-{%d + e, e) because of the shift- 
invariance. From the concavity, the condition O^ n < i& equivalent to the condition sup{z|fc(z) < 0} < 0, 
which implies that k{9) > 0. Thus, we have 

log .f{x,-e + e)- log /(a;, - - e) > 0. 
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Conversely, the condition 

Y,^ogfix,-0 + e)-\ogf{x,^d-e) >0 

i 

implies that k{9) > 0. Thus, we have sup{z|fc(z) < 0} < 9, which is equivalent to the condition O^^n ^ 0- 
Therefore, we have the relations 

Similarly, we can prove 

{fg\,iXn) > fe-ei^n)} C {0~„,. > 9} (Z {fe\,{x„) > /^.(xj}. 

Applying (|l8|), we can prove 

min{/3-(~40 + e,e),/?+(~i0-e,e)}= sup P ife-,\\fe+e), 

0<s<l 

which implies equation (p7[). ■ 
Corollary 1 When the function \ogf{x) is concave, the equation 

ai{9) = ai{9) 

holds. 

Lemma 2 When f{x) is monotonically decreasing, the estimator 9^ := {9^ :— 9^ — e} satisfies the relations 

/3+(£,0,e)= sup r{fe-,\\fe+e) 

0<s<l 

/?-(£, 0, e) = oo. (38) 

Thus, in this case, the equation 

ai{9) = ai{9) 

holds. 

Proof: Since 0„ > 9, we have „ > 9 — e, which implies (^). If 9^ „ > 9 + e, we have 9^>9 + 2e. Thus, 
f{xi — {9 + 2e)) > f{xi — 9) for any i = 1, . . . ,n. Therefore, 

fe+2A^n) > fe{xn)- 
Conversely, if fg_^2ei^n) > fei^n), we have 9n>9 + 2e. Thus, 

f0+2A£n\f^+2AXn) < f^^n)} = f^+2A^n\0^ <9 + 2e} = 0. 

Since the likelihood test {x„\9^ > 9 + 2e} achieves the optimal rate (|l^), we have 

lim-ilog/,"{f„|/,V(2'n) > f^iXn)} 

n 

= Km -1 log (/e"{^n|/e"+2.(^«) > fei^n)} + f'e\2e{^n\f^+2ei^n) < f^iXn)}) 

= sup r{fe\\fe+2e)^ sup r{fe-e\\fe+e)- 

0<s<l 0<s<l 



8 



Lemma 3 When the function x i— > log/(a;) is concave, MLE 9ml '■= {^AfL.n} satisfies that 

P+idML,9, e) = sup - log / exp ( -tQ^^^] f{x) dx (39) 



= sup - log / exp f-i^T^) /(^ + ^) '^^ (^0) 

t>0 J a \ J\^) J 

P- (Oml ,0,e)^ sup - log / exp (t^^:^^^) f{x) dx (41) 

t>0 Ja \ J(^ + '^)/ 

= sup - log / exp f ^4^) - ^) (42) 



This lemma is a special case of Fu's result |11 . For the reader's convenience, we give its proof. 

Proof: Equations (^) and (|2|) are trivial. We prove (|39|). From the assumption that for any a;„ := 
(xi, . . . ,Xn), the function d i— > log/l^^i ~ ^) is concave on {0{xn),d{xn)), the function 6 i— > ^"^j^ is 

monotonically decreasing, where /^(s;) := — y^fz^ on {9{xn),0{xn))- When x„ belongs to the support of fg', 
the condition 9ML.7i{xn) > ^' is equivalent to the condition 



1 " 

n ^ — ^ 



n 
1=1 



Denoting the conditional probability f{A\x G i?} under the condition x G i?, we can evaluate 

lim -i log f^iOML.n >e + e} = \mY-- log /^H^ml,„ > ^ + G (a + e, & + e)"} - - log fei^ + e, & + e)'^ 
n n n 



= lim--log/^;J >ol -log / (43) 



where the probability density function whose support is (a + e, 6) is defined by 

Chcrnoff's theorem (Theorem 3.1 in Bahadur |^) guarantees that 
1 1 " 

lim log fg ^{-y^le+t{xi) > 0} = sup - log / ex.p (tlg+eix)) faA^) dx 

n ' t>0 Ja+e 

= sup - log / exp f-t^-^^^) fix) dx + log / fix) dx. (44) 

Combining (^) and (p4[), we obtain (|39|). Similarly, We can prove (^ij). ■ 

Lemma 4 When fix) is monotonically decreasing, the MLE 0ML,n equals the estimator 6 

Proof: For any data x„ := (xi, . . . , a;„), if < ^„(a?'„)), /(xi — 6*) . . . /(a;„ — ^?) = 0. Conversely, if 
6* > O^ixn)), the obtained /(x, - 0) < fix., - ^„(f„)). Thus, ^„ is the MLE. ■ 

Lemma 5 Let f be a density function whose support is the interval (a, h). The estimators 9 and 6 satisfy 

/3+ ii 0,e) = - log fix) dx^ , /?- il 0,e)=oo (45) 

/3+(f,0,e) = w, p-it,e,e)^~\og( f fix)dx\ (46) 
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We can use the convex combination := {^(A)„ := A^,j + (1 — A)0„} with the ratio A : 1 — A, where < A < 1. 
It satisfies that 



/?+( (40,e) = -log(^y f{x)dxj (47) 

r('(4^,e)--log( r f{x)dx]. (48) 



Proof: Define tU„ :— maxjoji, . . . ,a;„},a;„ :— min{[x)i, . . . ,uJn}- Since the estimators 9,9, and (-^ are 
covariant for location shift, we may discuss only the case that 9 = 0. From the relation 9_^ > 9 > 9 — e, we 
obtain the second equation of (^5|). Its joint probability density function fnis^mfjln) given by 



/„(w„,ti;„) 
Defining 

we have 



n{n ~ 1) (XT* f{x) dx^ /(W„)/(W„) UJn > 



Un < UJ^ 



fix) dx UJn > W„ 



pUeWn <9^e)= f n{n- l).g(oz„,cJ„)"/(oz„)/(c^n) dw,, dS7„. (49) 

From the continuity of fijjJn) and g{LO_^,LJn), the equations 

lim ilogpg(0(A)„ < 6'-e) = lim - sup log5(w„,aJ„) 



= - log ( j f{x) dx 



hold. This implies the first equation of ( [45| ) and ([47|). In addition, we can similarly show the same for (^6|) and 

dH). ■ 



6 Two bounds in location shift family 

We discuss a location shift family generated by a probability density function (pdf) whose support is an interval 
(a, h). Moreover, we assume that the pdf / is continuous and satisfies that 

f{x)=Ai{x~aY^~^, x-^a + Q 
f{x)'^ A2{b-xY^-'^, x^b-0, 

where ni, K2 > 0, as for the uniform and beta distributions. 

When its support is a half line (0, oo) as for the gamma distribution and Weibull distribution, our situation 
results in the above case where = if / is continuous and lim^j^cxj l"^log/(^)l < Also, when 
Ki > K2, our situation results in the above case where A2 = 0. 

In the following, in this setting, we calculate two upper bounds ai{9) and a2{9), and derive a necessary and 
sufficient condition for coincidence of the two upper bounds. In addition, in some cases, we calculate the two 
bounds ai{9) and a2{9). 
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6.1 Semi-regular case 

As was proven in |q], when ki, K2 > 2, the relation 



holds, where this convergence is uniform for s and Jj is defined by 
Proposition 1 When g{x) — , we obtain k — 2 and the relation 



(50) 



Proof: Using (pOh, we have 



ai{e) = ai{9) = MO) = "2(0) = ^ Je. (51) 



ai(6') = 4^niax g = -Je 



o<s<i i^'" 2 

— //)^ • ^9-9 1 T 

a2(t') = mm — = -Jg. 

o<s<i s(l - ,s) 2 

When the function log/(x) is concave, using Lemma |^ we can evaluate 
/?+ (Oml ,e,e)>- log [ exp f -e^) /(^^ + e) dx 

J a 

rb-e 



/(^) 



iog(fV(.)4 



V /(^) 



log(l-r f(^)dx~^ [' ^ ^^dx + o{e^) 



b-e 2 A /(X) 

I 

/(4 



log ( 1 - ^ ( /'"^ dx - fib - e)] + oie') 



Thus, we obtain 



lim 1/3+ (.M., e) > i Ihn [ dx - /'(6 - .)) . 



Similarly, we can prove that 



€^0 e 

Since a2{0ML,0) < ^Jf, we have 



a2{0ML,d) = - J/, 



which implies (^l|). 
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6.2 The case that ki = K2 = 1 

As was proven elsewhere |lo|] |^ , when ki = K2 = 1 , the equation 

T -^"(/ell/e+e) . I /I /-I \ 
hm = Ats + A2(l — s) 

e^+O e 

holds, where this convergence is uniform for s G (0, 1). Letting g{x) — \x\, we have k = 1. 
Proposition 2 The relations 

aii8) = ai{e) = 2 sup g = 2 max{Ai, ^2} (52) 

0<s<l ' 

a2 (0) = a2 (9) = 2/|g = Ai + A2 (53) 
/loW. Therefore, ai{9) — a2{0) if and only if Ai — A2. 



Proof: The third equations of ( |52| ) and { p3\ j follow from the formula Ig g = ^is + ^2(1 — s). In the 
following, we prove the first equations of (|2|) and ( |5^ ) . By using Lemma ||, the equations 

hm V('(40,e) = 4^, hm 1/3- ('(4 



hold. Letting Ao := Ai/{Ai + A2), we have 

a2(''(A46') =Ai+A2, 



which implies the first equation of (p3[). 

Next, we prove the first equation of ( p2| ) in the case where Ai > A2. The estimator 6^ :— {9^ satisfies 



Therefore, 



/3+(£,0,e) = -log / Z?" (£, 0, e) = 00. 

Ja+2e 



lim -P(9„9,e) = 2Ai, 



which implies the first equation of (p2D. When A2 > ^1, we can similarly prove it. ■ 

6.3 The case that ki = K2 = 2 

As was proven by [p^ , when ki = K2 = 2 the equation 

hm . iA,+A2)sil-s) 

e^+o -e^logje] 2 

holds, where this convergence is uniform for s G (0, 1). Letting g{x) = —x'^logx, we have k — 2. Using (|5^), 
we have 



ai(0)=4 sup Il, = ^l±^ (55) 

0<s<l ^ 

q;2(6') = mf —-^ — - = . 

o<s<i s(l - s) 2 



Proposition 3 When the function x 1— > log/(x) is concave, we have 



ai (0) = Si (0) = a2 (0) - 52 (0) = ^l±ii . (56) 
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Proof: When a + S < x < b — S, we can approximate that 



exp — e 



/(^) 



f{a: + e) = f{x) + -(nx)- 



(57) 



Therefore, from Lemma |[ we can evaluate 



P+{9ML,0,e) 



> - log 
= - log 
= - log 



b-e 



exp — e 



a+5 
b-5 



exp — e 



/(x + e)dTj 

/(a; + e)(ix + y exp ^-e-^-!-^ + e) + ^ exp ^+e-^^^ 



(a; — e) dx 



a+5 

s 



a+S 



f{x)dx+ I f{x)dx+ / f{x)dx 

Jb-S 



Ai J ^exp (^~'^~^ + - dx + A2 ^exp ^+^~" 



(a; — e) — a; I da; + o(— log e) 



log 1 



^l±^e2log, + o(-,2loge; 



= -^4^^^1oge + o(-6^1og.), 

where the relation (|5^) follows from Lemma |ll|. Thus, we have 

a2\OML,0) = . 



(58) 



(59) 



Using and (|59|), we obtain ([5; 
6.4 The case that 1 < ki = K2 < 2 

As was proven by Hayashi pO[ |, when 1 < ki = K2 < 2, the equation 
lim 47^'(MI/e+.) 

Ais(l - s(ki - 1))B{S + Kl(l - S), 2 - Kl) ^2(1 - S)(l - (1 - s)(ki - 1))B(1 - S + KlS, 2 - Kl) 



holds, where this convergence is uniform for s e (0, 1), and B{x,y) is a beta function. Letting g{x) = \x\'^'^ , we 
have K = Ki. 

aiie) = 2- sup lie 

Q<s<l 

Ais{l - s{k - l))B{s + k(1 - s),2 - k) + ^2(1 - s)(l - (1 - s){k - 1))B{1 -s + ks,2- k) 



= 2" max 

0<s<l 

02(0) — inf 



g— + (1 - s)~ 



= inf 

0<s<l 



0<s<l s(l - s) 

Ais(l - s{k - l))B{s + k(1 - s), 2 - k) + ^2(1 - s)(l - (1 - s){k - 1))B{1 -s + ks, 2 - k) 



KS{1 — s) 



(s^ + (l-.s)^) 



K-1 



< -{Ai+A2)2^-\1-k)B 

K 



1 + K 



2 - K . 
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From Lemma we have 

lim —/?+(' (4 61, e) = Ai (60) 

lim ^/3-( "(40,-6) =^2—-^-—. (61) 

£^+oe«2'^ ^ ^ ^' ' ^ -K2(l-A)'*2 ^ ^ 

Thus, the estimator 9{X) achieves the optimal order. However, it does not achieve the optimal coefficient a2{&)- 
Proposition 4 //, and only if, Ai — A2, the equality 

holds. In this case, 

ai {9) = S2 [e) - — ^ > y 2 ^ i (g2 

K 

Proof: When Ai ^ A2, (|7|) and (|7|) of Lemma | guarantee that 



ds 



(Ai - Aa)^^^ — — ^TT tan i_^7rB(l^, 2 - k)s(1 - .s(k - l))B{s + k(1 - s), 2 - k) 7^ 0. 

(63) 



From the concavity and the continuity of the maximized function, we have ai{9) > 2'^!^ g. When Ai = A2, we 
have IgQ = e^- "^^^ relations 



1 yli2'*^-V3 - k)B (i±^,2 - k) 



L {0) = 2-^ J^^, = ^ '-^^ (64) 

follow from the concavity. Since the minimums 

min (s^ + (l-s)^) 
0<s<l V / 

and 

' (1 - s{k - l))B{s + k(1 - s), 2 - k) (1 - (1 - s){k - 1))B{{1 - s) + ks, 2 - k) ' 



mm , 

0<s<l \ 1 — s 

are achieved at the same point 5 = 5 (See Lemma ||), the relation 

, X K-i / (1 _ _ l))B{s + k(1 - s), 2 - k) (1 - (1 - s){k - l))B{(l - s) + ks, 2 - k] 



min fs"-! + (1 - s)"-!^ I 
0<s<l V / \^ 



0<s<l V / \^ 1 — s 

=2«-i(3-«)B(^i±^,2-^^ (65) 

holds. Thus, equation (^) follows from (|6^) and (p5|). ■ 

Next, we consider the case A2 = Q. 

Proposition 5 When 1 < k < 2 — ig, relations 

2'^ 

ai{e) ^ai{e) = Ai— (66) 

/loZd, where the real number io S (0, 5) is uniquely defined by (see Lemma 

2ta + tail - to) (V^(l + to) - ^(1)) = 1, (67) 
where i>{x) is the D-psi function defined by ^l^{x) := logT(a;). 
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The number to is enumerated by to = 0.432646, as is checked by the following graph. 

Proof: In this case, since the function s i— > s(l — s{k — l))B{s + k(1 — s),2 — k) is concave, Lemma ^ 
guarantees that 

-4- s(l - sU - l))B(s + k(1 - s), 2 - k)) > - s{k - l))B{s + k(1 - s), 2 - k) 

as as 

= (3 - 2k) + (2 - k){k - 1)(V'(3 - k) - V(l)) > 



for s e (0,1). Thus, 



_ ^12" 

o^i{&) — max s(l — s{k — l))B{s + k(1 — s), 2 — k) 

= _ 1(k - 1))B{1 + k{1 - I), 2 - k) = 



As in subsection 



6.2, we can prove that 



1 ^ At2'^ 



which implies 
6.5 The case that < /ti = /«2 < 1 

As was proven by Hayashi |l^, when < ki = K2 < 1, the equation 

I"{fe\\fe+e) ^ l:zJlLfAisB(s + ki(1 - s), 1 - ki) + ^2(1 - s)B(l - s + ms, 1 - m)) 

holds, where this convergence is uniform for s G (0, 1). Letting g(x) = x^, we have k = Ki. 
ai(e) = 2^^ max I'g 

0<s<l 

_ , AisB{s + k{1-s),1-k) + A2{1-s)B{1-s + ks,1-k) 

0<S<1 K 

MO)^ sup fs^ + {I - s)^y 

0<S<1 s[l - s) \ J 

, AisB(s + k(1 - s), 1 - k) + ^2(1 - s)-B(l - s + Ks, 1 - k) / _L , , 1 
= sup (1-k) ^ ( s-1 + (1 - s)"-! 1 



Proposition 6 //, and only if, Ai — A2, the equality 

ai{e) = a2{0) (68) 

holds. In this case, the equation 

Si(e) = MO) = ^A,2'' {1~k)B 1 - (69) 

holds. 

Proof: Using ( [76| ) and (|7^) of Lemma we obtain 



^ = (Ai - A2)^-^COt -—ttB{^-, 1 - At). 

S 7j 



Since -!-^7r cot ^-^ttB{^^^, 1 — k) > 0, Theorem ^ yields this sufhcient and necessary condition for (|68|). 



Equation (hq) implies (|69|) 
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However, since the function x t— > (k — 1) logx is convex on (0, oo), the function x i—>- log/(x) is not concave 
on (a, 5). There does not exist an example in which Lemma H can be apphed. Thus, it is an open problem 
whether there exists an example such that 

in this case, except for the case A1A2 = 0. 

Proposition 7 When Ai = 0, ol\{Q) and ai{Q) are calculated as 

ai{0)=ai{9) = ^^ (70) 

K 

a2(^)=«2W = — . (71) 

K 

Proof: Since the function s h-> sB{s + k(1 — s), 1 — k) is concave, we have 

-^sB{s + k{1 - s), 1 - k) > -^sB{s + k{1 - s), 1 - k) 
ds ds 

= l + (l-«)(^(l)-^(2-«)) 

> 1 + (1-«)(^(1) -7/^(2)) (72) 
= 1 - (1 - k) = K > 0, (73) 

where inequality ( [72[ ) holds because is monotonically increasing in a; G (0, cxd), and the first equation of 
(73) follows from the formula tp{x + 1) = ^j{x) + K Thus, 

ai(e) = — — ^ max sB(s + k 1 - s), 1 - k) 

K 0<S<1 

= — — ^ -B(l, 1 - K = ^— . 

K K 

Since, as in subsection |6.4 we can check that the estimators {0^}e>o achieve the bound equations ( [fo| ) 

hold. 

The other upper bound a2{0) is calculated as 
Alii ~ k) 

02(6*) = — max B{s + k(1 - s), 1 - k) 

K 0<s<l 

Note that the beta function B{x, y) is monotonically decreasing for x, y > 0. Since both maxo<s<i B{s + k(1 

/ 1 

s), 1 — k) and maxo<s<i I ("^^) ^ " + 1 ) are achieved at the same point, s = 1, we have 

a2{0) = 5(1, 1 - k) = — . 

K K 



This bound is achieved by the estimator 6_ because 

1 ^ —1 A 

lim — (3(0,9, e) — lim^ — log / f{x)dx 

Therefore, we have (iTll 



7 Conclusion 

We have discussed large deviation theories under a more general setting. The two quantities ai{9) and 7x2(0) 
do not necessarily coincide. In a non-regular case, it is clear that the order of limits is crucial. In the future, 
such phenomena deserve study from another viewpoint. 
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Nagaoka |l^ initiated a discussion of two kinds of large deviation bounds, as in this paper, in a quantum 
setting, and Hayashi discussed these in more depth. The two kinds of large deviation bounds do not 
necessarily coincide in a quantum setting. However, the reason for this difference in a quantum setting differs 
from that for a non-regular setting. Gaining an understanding of these differences from a unified viewpoint 
remains a goal for the future. 



A Lemmas concerning the beta and D-psi functions 

In this section, we prove some formulas concerning the beta and D-psi functions used in Section ^. 
Lemma 6 When 1 < k < 2, we have 
d 



ds 



s(l - s{k. - l))B{s + k(1 - s), 2 - k) 



— (1 - s)(l - (1 - s){k ~ - s) + KS, 2 - n) 

ds 



s=l/2 



s=l/2 



(k-1)(3-k) 2-k + 

^ '-^ ^TTtan nB( ,2-k) >0 

4 2^2-^ 



(k-1)(3-k) 2-k A+k 

^ '-^ ^TTtan ttB{^—,2~k) <0 

4 2^2^ 



(74) 
< 
(75) 



When < k < 1, the equations 



—sB{s + k(1 - s), 1 - k) 
ds 



-(1-s)B((1-s) + ks,1-k) 



s=l/2 



= 1/2 



1 — K 1 — K + K 

TTCOt -KBi , 1 - k) > 

2 2^2'-^ 

1 — K, 1 — K + K 

TTCot 7rB( ,1-k) <0 



hold. 



(76) 
(77) 



Proof: Using the function :— -^logr(a;), we can calculate 
d 



ds 



s(l - s(k - l))B{s + k(1 - s), 2 - k) 



--1/2 



{2-k) + 
{2-k) + 
{2-k) + 



(3- 




-k) 




4 




(3- 




-n) 




4 




(3- 




-n) 



^(- 



5 — K. 



, , K 1, 2 .3 — hh . 2 

i'( ) H iPi ) 

2 ^ K-1 2 ' 3-K 



TT cot TT- 



3 — K 



4k 



(k-1)(3-k) 



B(^,2-.) 



(k - 13 -k) 2-k^,1 + k„ 

'-^ ^TTtan nB( ,2 - k) 

4 2^2'^ 



(78) 
(79) 
(80) 



where we use the formula ^p{x + 1) = ^ -I- ipi^) in (Kg ), the formula ^p{l — x) ~ ip{x) — TrcotTrx in ([79|), and the 
formula cot(^ -f x) = — tana; in (|^. We obtain <^^- Similarly, we can prove ([75|). 
Next, we prove (|76|). We can calculate 



— sB{s + k(1 - s), 1 - k) 
as 



= 1/2 



1 — k/,/1 + k, ,/3 — kA\„,1 + k 

= 11+— « — )-* — ) «(— 



1 — K 1 — K 1 + K 

— ^vrcot^— 7rB(— ^,1 - k), 



5(^,1--) 



(81) 
(82) 
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where (|8l] ) follows from the formula i/j{x + 1) = j + ip{x) and ( p^ ) follows from the formula — x) — ipix) = 
TTCotTra;. Similarly, we obtain (|77|). ■ 

Lemma 7 Assume that 1 < k < 2. There uniquely exists tg £ (0, 5) satisfying ^6^, and the inequality 

(3 - 2k) + (2 - - 1)(V'(3 - k) - t/-!!)) > (83) 

holds if, and only if, 1 < k < 2 — t^. 

Proof: In the following, this lemma is proven by replacing k with 2 — t. Define the function h{t) := 
2t~l + t{l- + t)- ipil)). When i < i, we have 

h'{t) = 2 + (1 - 2f)(V'(l +t)- V^l)) + {t- 2t^)'ijy{l + t) > 

because ip'{x) > for x > 0. Therefore, h{t) is strictly monotonically increasing in (0, Since h{0) = — 1 < 
0, h{^) = J — ipi'^)) > 0, there uniquely exists the number to S (0, ^) satisfying (|67|). Also, in this case, 

the inequality h{t) > holds ii t > tg. 

Next, we consider case t > ^. Given the relations, 

2t-l>0, i(l"t)>0, + t) - > 0, 

h{t) > 0. ■ 

Lemma 8 The minimum 

. / (1 - s(k - l))B{s + k{1 - s), 2- k) (1 - (1 - s){k - 1))S((1 - s) + ks, 2 - k) 
0<S<1 \ 1 — s s 

is attained at s — ^. 

Proof: Since the minimized function is invariant for the replacement s > 1 — s, it is sufficient to show its 
concavity. Since -^e''''^'' = {h"{x) + (/i('a:))^)e''^^\ we can show its concavity by proving the concavity of the 
function s log ^—j^^^^Bls + k(1 — s), 2 — k). We can evaluate 

— ^ logB(s + k(1 - s), 2 - k) = (1 - nf {'4}'{s + k(1 - s)) - ijj'is + n{l - s) + 2 - n)) > 

because '0'(a;) is monotonically decreasing for x > 0. Also, we have 

d^ I-s(k-I) _ (2-k)(k-2(k-1)s) 

1^^^ " (1 + ks + (k-1)s2)2 > ^ 

because k > 2{k — 1). The proof is now complete. ■ 

B Concave function 

Lemma 9 When a concave function f > is defined in (0, 1), 

. -tx + fit) (^s-t)x + {l^s)f{t) 

misx + (1 - s) sup — = mf sup = /(s). 

x>0 0<t<l 1 — t 3:>Oo<t<l i— t 

Proof: Substituting s into t we have 

0<t<l 1 — t 

Taking the infimum infa;>o, we obtain 

^ ^ . . is-t)x+il-s)fit) 

f{s) < mf sup . 

^>Oo<t<i t — t 
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Next, we proceed to the opposite inequality. From the concavity of /, we can define the upper derivative /' 
and the lower derivative /' as 

Tis) lim M-Iil^, llllll^ 

Since the concavity guarantees that 

e — e 

we obtain 

is - s)if{s) + (1 - s)n-s)) + (1 - s)fi.s) (s - t){fis) + (f - s)ns)) + (1 - ■s)fit) 
1-s 1-t 

=T~1 (^^'^ ~ ^^^^ + " - ^ 

Therefore, 

, ^ {s-t){fis) + {l-,s)Tis)) + il-s)f{t) . (^-t)(/Gs) + (l-.)ns)) + (l-.)/(t) 

/(s) > sup > nif sup . 

o<t<i 1 — r ^>Oo<t<i i — I 

The proof is now complete. ■ 

C Other lemmas 

Lemma 10 When g is strictly monotonically decreasing and continuous and satisfies that g(0) — 0, there exists 
K > such that 

x^= lim X >0. (84) 

.-+0 5(e) 

Proof: Let h{x) be the RHS of (||). Since 

lin, ^ ^ lini hm 
e^+o g{e) £-+0 g{ye) e-+o g{e) 

h{xy) = h{x)h{y). Thus, there exists k > satisfying (p^. ■ 

Lemma 11 For any 5 > 0, we have 



I f l\ 1 

lim / exp — e— (x + e) — a; = - 

e-0e2 1oge7o V ^/ 2 

1 / 1\ 1 

lim -r- / exp +e— (x — e) — a; da; = -. (86) 



(85) 



Proof: we can calculate 

r<5 oo T,<r"+1 



n{n-l) /e"+i 



/ exp(^+.-j(.-e)-.da: = y^ g - (^^^1)!^ ^ " 2 ^^"^^'^ " ^^^e) + g ^ ^^„_, 
Since ^ — > 0, we obtain (p6|). Similarly, we can calculate 



20 



Since 



we obtain (|85|). 

Lemma 12 Jbr < s < 5, i/ie inequalities 

2slHp\\q)<r{p\\q)< 2(1 -s)lHp\\q) 

hold. 

Proof: Since (^) ^ + (1^9;) = Ij the Holder inequality guarantees that 

f{Lo)q'-^ico)dco= [ {p%Lo)q^Lu)) {q'''%L0)) diO 

n Jn 

< {p^{u;)q%co))^ dco^ ■ ^ (q'-^^cu))^ dco^ 

p^(uj)q^{uj) duj j 
n J 

Thus, we obtain 

rip\\q)>2slHp\\q). 

Similarly, since (2(1 — s))~^ + f-[^i^) = 1, we can apply the Holder inequality as 



p^Lu)q^uj) duJ ^ / {p''{uj)q^~'{uj))^''-''^ p^^{uj)duj 
Jn 



, 2(1- = ) 

p^{Lo)q {lo) dio 



which implies that 



lHp\\q)>—^J\p\\q). 
2(1 -s) 
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